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Abstract 

Measuring the Higgs self-coupling is one of the crucial physics goals at the LHC Run-2 and 
other future colliders. In this work, we attempt to figure out the size of SUSY effects on the 
trilinear self-coupling of the 125 GeV Higgs boson in the MSSM and NMSSM after the LHC Run- 
1. Taking account of current experimental constraints, such as the Higgs data, flavor constraints, 
electroweak precision observables and dark matter detections, we obtain the observations: (1) In 
the MSSM, the ratio of Dear Oe has been tightly constrained by the LHC data, which can be 
only slightly smaller than 1 and minimally reach 97%; (2) In the NMSSM with A < 0.7, a sizable 
reduction of A SoM y ASM can occur and minimally reach 10% when the lightest CP-even Higgs 
boson mass Mp, is close to the SM-like Higgs boson mp, due to the large mixing angle between 
the singlet and doublet Higgs bosons; (3) In the NMSSM with à > 0.7, a large enhancement or 
reduction —1.1 < ASRM / ASM < 2 can occur, which is accompanied by a sizable change of 
hir*7~ coupling. The future colliders, such as the HL-LHC and ILC, will have the capacity to 


test these large deviations in the NMSSM. 
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I. INTRODUCTION 


With the discovery of the Higgs boson at the LHC Hl d, much effort has been devoted 
to study its properties. So far, the measurements of its couplings and quantum numbers are 
compatible with the standard model (SM) predictions at 1-2ø level. However, to ultimately 
understand its nature, we need to fully reconstruct the Higgs potential at the LHC and 
future ete” colliders . The parameters in the Higgs potential determine the relations 
among the Higgs masses and self-couplings. Measuring these relations is therefore crucial 
for our understanding of the Higgs nature. 

In the SM the tree-level Higgs potential is given by 


VOSM) — _1?(9td) +AA, d= z (0,v +A)” (1) 


which yields the following trilinear and quartic self-couplings 


Age) = Sk, apao = TE © 
Here v = (vV2Gpr) 7! œ 246 GeV is the vacuum expectation value of the Higgs field and 
mp ~ 125 GeV is the Higgs boson mass. Within the SM, the trilinear Higgs coupling receives 
the dominant correction from the top quark loop, AFM ~ m4/(1?v?m?) f which reduces its 
tree-level value by about 10%. Hence, the determination of the Higgs trilinear coupling Anah 
and quartic coupling Anaan can directly test the relation in Eq. which is obtained from the 
minimization of the Higgs potential. At the LHC, the only way to measure the Higgs trilinear 
coupling is through the Higgs pair production, which is dominated by the gluon fusion 
mechanism and has a small cross section [|6]. However, in many new physics models, such as 
the minimal supersymmetric model (MSSM) and the next-to-minimal supersymmetric model 
(NMSSM), the Higgs pair production rate can be significantly altered by new particles and 
Higgs couplings i Among various decay channels of the Higgs pair, the 4b final state 
has the largest fraction ld) but the rare process hh — bbyy is expected to have the most 
promising sensitivity due to the low backgrounds at the LHC |9]. The recent applications of 
jet substructure and other techniques to Higgs pair production have been found to improve 
the sensitivity to the trilinear Higgs couplings in 7*r~ and WtW- final states 10). On 


the other hand, the measurement of the Higgs quartic coupling is more challenging due to 


a much smaller cross section of triple Higgs production at the LHC. 


The supersymmetric (SUSY) corrections to the Higgs self-coupling have two kinds of 
sources: one is the mixing between the Higgs bosons, and the other is radiative quantum 
effects. For the first kind, the authors in studied the Higgs self-coupling in some simpli- 
fied SUSY models, while in [12] the Higgs self-couplings in the MSSM and NMSSM (with 
a decoupled singlet boson) were investigated. Also, E: the authors studied the proper- 
ties of the Higgs bosons in the NMSSM with A > 0.7 (called A-SUSY) and found a sizable 
enhancement in the Higgs self-coupling. For the second kind of SUSY corrections, the loop 
corrections to the Higgs self-couplings have been studied using the effective potential 
or Feynman diagrammatic approach in the MSSM and NMSSM. Recently, the 
leading two-loop SUSY-QCD corrections from the top/stop sector in the MSSM have been 
performed ld. Since all these previous studies are limited to some simplified or special 
cases and the relevant experimental constraints are not fully considered, in this work we 
give a comprehensive study for the SM-like Higgs self-coupling in the MSSM and NMSSM 
with both A < 0.7 and A > 0.7 by considering all the relevant experimental constraints after 
the LHC Run-1. 

The existing experimental data, both from low energy precision measurements and high 
energy direct searches, may have imposed important constraints on the Higgs self-coupling 
in SUSY models. For example, in the MSSM, the SM-like Higgs self-coupling is sensitive 
to the pseudo-scalar mass m4 and tan 3, which could have been tightly constrained by the 
LHC direct search of a light non-SM Higgs boson as well as from precision B-physics 
30]. Furthermore, the measured mass of the SM-like Higgs boson requires rather heavy 
stops and/or large Higgs-stop trilinear couplings, and the direct searches for the stop pair 
production have also pushed stop masses above hundreds of GeV in the natural SUSY m 
Since the Higgs self-couplings are sensitive to stop masses and Higgs-stop trilinear couplings, 
all these constraints should be taken into account. 

The structure of this paper is organized as follows. In Section O| we will briefly describe 
the Higgs sectors of MSSM and NMSSM. In Section [I] we perform a scan over the parameter 
space of each model and present the numerical results for the trilinear self-coupling of the 


SM-like Higgs. Finally, we draw our conclusions in Section 


Il. HIGGS TRILINEAR SELF-COUPLINGS IN MSSM AND NMSSM 
A. Higgs trilinear self-couplings in the MSSM 


In the MSSM there are two doublets of complex scalar fields with opposite hypercharges: 


H% H} 
— ; Hı = ; (3) 
H? Hy 
The scalar Higgs potential consists of the D-terms and F’-terms of the superpotential as well 
as the soft SUSY-breaking mass terms. Among them, the D-terms determine the quartic 
Higgs interactions. The full tree-level Higgs potential is given by 
VOMSSM) = m? A? + m3|Hal? — Beas HEH} + h.c.) 
g? +g’? 
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where €ag is the antisymmetric tensor and Mm? = m?, +p? with my, , and u denoting the 
B 1,2 Hua TH ud H 


+ 


2 
(|Hu? — |Hal?) + S [HL Hal?, (4) 


soft SUSY-breaking masses and the higgsino mass, respectively. The parameters mı, can 
be eliminated by the minimization condition of the Higgs potential, while the parameter B, 
is traded for the pseudoscalar mass M4. The quartic Higgs couplings are fixed in terms of 
the SU(2) x U(1) gauge couplings g and g’ in the MSSM. 

After the electroweak symmetry breaking, the neutral components of the two Higgs fields 
H? q develop vacuum expectation values (vevs) v,,q and can be decomposed into scalar and 


pseudoscalar components as 


Re H} = (va+ Hea —hsq)/V2, Im HY = (G°eg — Asg)/V2, (5) 
Re H? = (vu, + Hsq + hea)/V2, Im H? = (G°sg + Acg)/V2 (6) 


where h, H and A are the neutral physical Higgs bosons and G® is the would-be Goldstone 
boson. The vevs are defined as v, = vsg and vg = veg with v ~ 246 GeV (here and in the 
following we use the notation Cy = cos x, s, = sin z). 

Taking the third derivatives of V©MSSM) with respect to the physical Higgs fields yields 
the trilinear Higgs couplings. In the physical mass eigenstates, the neutral CP-even Higgs 


trilinear couplings at leading order are given by 


0,MSSM 3M? omssm) M3 
A a Fr f OraSatps Ate ie — [28208048 = CreCa+el; (7) 
3M2 M2 
AA R = v Z coelatpy Meee z= ——2 [252aCa48 + C2aSa+8]: (8) 


In the MSSM, either the lighter scalar h or the heavier scalar H can be the SM-like Higgs 
boson. The latter interpretation occurs for low values of M4 (between 100 and 120 GeV) with 
moderate values of tan 6 (about 10). In this case, H has approximately SM-like properties, 
while the other four Higgs bosons of the MSSM would be rather light and have a mass of 
order 100 GeV or even below. A dedicated scan for this region of parameter space has been 
performed in and it was found that this scenario can be excluded by recasting the LHC 
search for H/A —> tt77 . In addition, the latest ATLAS limits from H= searches have 


also excluded such a possibility [29]. Thus, in this work, we will only study the case that 
h is the SM-like Higgs boson in the MSSM. According to Appelquist-Carazzone decoupling 
theorem, the MSSM must go back to the SM in the decoupling limit. In the tree-level Higgs 
sector, we can make this limit by setting ma — oo, which gives a + 6 — 5. Applying this 


relation to the first identity in Eq. (Z) yields 


MSSM) 3MZ 2 3M}, (0, MSSM) 9 
hhh =~ —— C = ——-—- —e (9) 


where Mn,o,mssm) ~ Mzcag is the lighter CP-even Higgs mass at tree-level. This demon- 
strates that the lighter Higgs boson h in the MSSM almost behaves like the SM Higgs boson 


in the decoupling limit (even when the loop corrections are included) |23, |24]. 


B. Higgs trilinear self-couplings in the NMSSM 


After the discovery of a 125 GeV Higgs boson, the NMSSM seems to be more favored 
than the MSSM because it can naturally give such a Higgs boson without very heavy top- 
squarks [34]. More importantly, this model can solve the y-problem: after the singlet field 
develops a vev (S) = v,/\/2, an effective p-term (eff = Avs/V2) is dynamically generated. 
Due to the contribution of the singlet scalar field S, the full tree-level Higgs potential can 


be written as 


VONMSSM) _ (\AS? + mi, HLH, + (\AS? + mi, H) Ha + m3|$\? 


1 1 
+3 (9 + 93)(HiH, — H} Ha)? + 5 92| i Hal 


1 
+e? HOH? + K322 + [e AA HeHS + zrAns® + h.c], (10) 


where « and à are dimensionless parameters, and A, and A, are the corresponding trilinear 


soft breaking parameters. To clearly show the properties of the Higgs sector, we can expand 
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the neutral scalar fields around the vevs as 


ReH? = (va—Hsin6+hcos$)/V2, Im H? = (P sin 8 + G° cos 8) /V2, 
ReH? = (v, + Hcos8 + hsin B)/V2, Im H? = (P cos 8 — G° sin 8)/V2, 
Im § = Ps/V2. 


ReS = (v, + s)/V2, 


(11) 


Substituting Eq. into Eq. (10), we obtain the mass matrix squared M2 for the neutral 


CP-even Higgs bosons and the trilinear Higgs self-interactions as 


H 


o = 2 (H, h, s) M3 h + Ahahghy Aghghy ’ 


S 


with hagy = H, h, s. The tree-level M2 ij are given by 


1 
Mou = M3 + (Mz = JA) sin? 26, 
1 


1 
Mig = = — 5 v) sin 48, 


M2,, = —V2dvpx cot 26, 

1 
Meo, = M3 cos? 26+ a sin? 28, 
Mš» = V2dop(1 — 2), 


( 


K 
Ay + 2—u 


k? K My? KA o. 
Meg = Sva + JAH + a Eu sin 28, 
with 
Av 1 
M2 = = (v2A .) ae 
A sin 28 V2A, +e), z 2u 


A 


). 


(19) 


Here, it should be mentioned that the mass parameter M4 in the NMSSM becomes the mass 


of the heavy pseudoscalar Higgs boson only in the MSSM limit (A, x — 0 with the ratio K/X 
fixed). In the NMSSM, M4 can be traded by the soft parameter A). 


The CP-even Higgs mass eigenstates h; (i = 1,2,3) can be obtained by diagonalizing M2 


with a rotation matrix O 


y= Ohar a= His), deem o Mar] = 0M 0 


with Oia being the elements of the rotation matrix satisfying the sum rules 


Ola + O34 + O3, = 1- 
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(20) 


(21) 


The mass eigenstates h; are aligned by the masses Mma, < mp, < mp. The singlet or non- 
SM doublet components in a physical Higgs boson h; is determined by the rotation matrix 
elements Ojy,s). With Eq. (20), the corresponding tree-level trilinear Higgs couplings in the 


mass eigenstates h; are given by 
(0,NMSSM) 
Nicely = OiaOjBOkyAhahghy: (22) 


In the NMSSM, we take h, or ho as the SM-like Higgs boson when lOa] > 0.5. In 
general, due to the introduction of the singlet s and its couplings to the MSSM Higgs sector, 
the mass of the SM-like Higgs boson M4, can be lifted by the extra large A-term at tree 
level, as shown in Eq. (16). The value of À at the weak scale is upper bounded by 0.7 in 
order for the NMSSM to remain perturbative up to the GUT scale led Lah. Whereas, the 
case of A > 0.7 (dubbed as A-SUSY model) is still of interest because it can suppress the 
sensitivity of the Higgs mass with respect to changes of the soft SUSY-breaking masses and 


keep the fine tuning at a moderate level even for stop masses up to 1 TeV. So, in our study, 


we consider both A < 0.7 and A > 0.7 cases: 


e For  < 0.7, in addition to the tree-level À contribution, the mixture of the singlet 
s with the MSSM Higgs h and H, as shown in Eq. (13), in particular with h, could 
further modify the SM-like Higgs mass. If H is decoupled, when Må, > Mŝ,,, the 
mass eigenvalues for the SM-like Higgs boson mp, is pushed up by the positive mixing 
effect after the diagonalization of the h — s mass matrix. However, when Må, < Må, 
the mass eigenvalues for the SM-like Higgs boson mp, is pulled down by the negative 
mixing effect. Without the negative mixing effect, the maximal tree-level SM-like 
Higgs mass mp, can only reach to about 110 GeV Lad ld, which needs a sizable loop 
correction from the stop sector to obtain a 125 GeV Higgs boson like in the MSSM. In 
this sense, hy being the observed SM-like Higgs boson may be more natural than hı in 
the NMSSM. So, we choose the next-to-lightest CP-even Higgs boson hz as the SM-like 
Higgs boson in the following discussions for the NMSSM with A < 0.7. Note that the 
lightest Higgs boson h; in this case is predominantly singlet-like and its mass can be as 
light as about 20 GeV in our scanned samples. Consequently, the SM-like Higgs boson 
hə can decay into a pair of light scalars hı and hence the yy and ZZ* signal rates are 
suppressed. In order to be consistent with the LHC ba. data, the branching ratio 

38 


of hə + hıhı was found to be less than about 30% and may be tested through 


T 


hıhı —> bbu” production channel at the 14 TeV LHC lao). However, due to our 
interests in the large mixing region, we only display the results with Ma, > mp,/2 in 
the following analysis. We will also decouple the heaviest CP-even Higgs boson h3 by 
requiring Mp, > 1 TeV and focus on the singlet-doublet system. On the other hand, as 
mentioned above, if hy is the SM-like Higgs boson, the value of \ tends to be large in 
order to maximize the tree-level Higgs mass (so in our study for A-SUSY with à > 0.7, 
we will choose hı as the SM-like Higgs boson). This feature may lead to a sizable 
change in Higgs self-coupling when doublet-singlet mixing effect is large. We checked 
this possibility and found that the ratio A SSM /\3™ can vary from 0.29 to 1.17 for 


our samples with A < 0.7. 


e For à > 0.7, we choose the lightest CP even Higgs boson A; as the SM-like Higgs boson. 
The reason is that for A > 0.7 the tree-level Higgs mass will be significantly lifted. If 
hy is assumed as the SM-like Higgs boson, the large A-term in M3,, and the positive 
doublet-singlet mixing effect can readily make the SM-like Higgs mass mp, exceed 125 
GeV. Thus, such a choice is strongly disfavored by the LHC observed Higgs mass 
and will not be further studied in our work. If hı is the SM-like Higgs boson, as pointed 
before, the cancelation between the tree-level A-term and the negative doublet-singlet 
mixing effect can easily yield a 125 GeV SM-like Higgs boson. However, A can not be 
too large, because the large trilinear and quartic couplings of singlet scalar field s will 
largely contribute to the scattering amplitude ss — ss. In the high energy limit, the 
unitarity condition requires |A| < 3 and |x| < 3. Furthermore, if combined with the 
dark matter relic abundance constraint, the above-mentioned unitarity bound can set 
a generic upper bound 20 TeV for the heavy Higgs masses |40. The requirement of 
perturbativity up to the cut-off scale A, i.e., A(A) S 27 and K(A) S 27, will set upper 
bounds on A and « at weak scale. In this study, we assume the new unknown strong 
dynamics, for restoring the unitary of la processes, appears at some scale A 

41 


above 10 TeV, and require \? + K? < 4.2 35| 


Before closing this section, we note that since both Higgs boson masses and Higgs self- 
interactions arise from the Higgs potential, one has to adopt the same method to calculate 
them up to the same order for comparing the self-couplings in SUSY and SM. Although the 


most accurate evaluation of Higgs mass is up to three-loop level in the MSSM |42), there is 


no corresponding result for the Higgs self-couplings. Hence, in our study, we derive the Higgs 
mass and self-couplings by following the effective potential approaches in the MSSM |14 
and NMSSM [21], where the explicit dominant one- and two-loop corrections to the effective 
potential are presented. Then we coded those expressions in our numerical calculation. On 
the other hand, since the tree-level mixing effects in the Higgs sector are usually dominant 
over the loop corrections in the NMSSM, we will focus on the doublet-singlet mixing effects 


for the NMSSM results. 


Il. NUMERICAL CALCULATIONS AND RESULTS 
A. Scan over the parameter space 


In our numerical calculations, we take the input parameters of the SM as 


m = 173.5 GeV, mw = 80.385 GeV, mz = 91.1876 GeV, 
mM5 (mM) = 4.18 GeV, a,(Mz) = 0.1184, a(mz)~! = 128.962 (23) 


We use NMSSMTools-4.4.1 to perform a random scan over the parameter space. Note 
that for any given value of ery, the phenomenology of the NMSSM is identical to the MSSM 
in the limit A, « — 0 with the ratio K/A fixed (A, should be negative and satisfy |A,| < 4Ku/A 
to guarantee the squared mass of the singlet scalar to be positive) lhg. In our scan for the 
MSSM, we take \ = «x = 1077 and A, = —10 GeV. The validity of such a method has been 
justified by the authors of the NMSSMTools and our previous calculations . We also 
numerically checked our MSSM results of m, by using the codes FeynHiggs lash SOFTSUSY 
and SuSpect lad, and found the results to agree with that given by the NMSSMTools 
within about 1% level when mp ~ 125 GeV. So it is feasible to use the package NMSSMTools 
in the MSSM limit to study the phenomenology of the MSSM. For simplicity, we decouple 
the sleptons, gauginos and the first two generations of squarks by fixing the corresponding 
soft mass parameters at 2 TeV. We also set Mg3 = Mp3 = My3 and A; = A, for the third 
generation of squarks. The lower limit of tan 8 in the MSSM is taken as 5, which is inspired 
by the recent LHC Higgs results |32]. The parameter ranges in our scan are chosen as the 


following: 


(a) 


(b) 


(c) 


For the MSSM, 

5<tan8 < 60, 0.2 TeV < My <1 TeV, |u| <1 TeV, 

0.1 TeV < Mos < 2.5 TeV, |A| < 3MQ3. (24) 
For the NMSSM with A < 0.7, 


0<A\<0.7, |x| < 0.7, 0.2 TeV <A <1 TV, 
|A| < 1 TeV, 1<tan6 < 20, 0.1 TeV < u< 1 TeV, 
0.1 TeV < Mos < 1 TeV, |A| < 3 Mos. (25) 


For the NMSSM with A > 0.7, 


O7<\<2 0% <9, 02 TeV < A, <1 TeV, 
|A,| <1 TeV, 1 < tanp < 20, 0.1 TeV < u <1 TeV, 
0.1 TeV < Mg3 <1 TeV, JA] <3Mos. (26) 


In our scan, we consider the following constraints: 


(1) 


We require the SM-like Higgs mass in the range of 123-127 GeV and consider the 
exclusion limits (at the 95% confidence level) from LEP, Tevatron and LHC in Higgs 
searches with HiggsBounds-4.2.0 |48]. We also perform the Higgs data fit by calculating 
y? of the Higgs couplings with the public package HiggsSignals-1.3.0 |49] and require 
our samples to be consistent with Higgs data at 20 level. We choose the SLHA 
input choice of HiggsBounds/HiggsSignals, where the effective Higgs couplings are 
only used to calculate the Higgs production cross section ratios. The Higgs decay 
branching ratios are taken directly from the corresponding decay blocks in the SLHA 
file generated by the NMSSMTools. 


We require one-loop SUSY predictions for B-physics observables to satisfy the 20 
bounds as encoded in NMSSMTools, which include B > X,y, Bs > pty, By > 
Xut and Bt — rtv. Theoretical uncertainties in B-physics observables are taken 


into account as implemented in NMSSMTools. 


We require the one-loop SUSY predictions for the precision electroweak observables 
such as p;, sin? Olg, my and R, |50] to be within the 2c ranges of the experimental 


values. 
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(4) We require the thermal relic density of the lightest neutralino (as the dark matter can- 
didate) is below the 2g upper bound of the Planck value and the spin-independent 
neutralino-proton scattering cross section satisfy the direct detection bound from LUX 
at 90% confidence level 


(5) We also consider the theoretical constraints from the stability of the Higgs potential 
as encoded in the NMSSMTools. 


B. Results for the MSSM 


nds-4.2.0 $ 


MA 


FIG. 1: Excluded region in the tan 8 versus Ma plane of the MSSM. 


Precision measurements of the Higgs boson properties (its mass and couplings to other 
particles) at the LHC provide relevant constraints on possible weak-scale extensions of the 
SM. In the usual context of the MSSM, these constraints suggest that all the additional 
non-SM-like Higgs bosons should be heavy. In Fig. [| we show that in the decoupling limit 
of the MSSM, the region with m4 < 330 GeV has been excluded by various constraints. 
Similar result has been recently pointed out in ba. The lower part of Fig. [1] shows that 
the value of tan can not be too small due to the requirement of 125-127 GeV Higgs mass 
in our scan. For a small tan, heavy stops (2 10 TeV) or a large mixing parameter A; 
is needed to produce a large positive correction to the Higgs mass, which, however, will 
easily lead to vacuum instability and large uncertainty in the numerical calculation. So, we 


focus on mz, < 2.5 TeV region in our calculations. The additional constraints on the MSSM 
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parameters imposed by the Higgs data are obtained by using the HiggsBounds-4.2.0 package. 
The key algorithm of HiggsBounds can be described in two steps. Firstly, the HiggsBounds 
uses the expected experimental limits from LEP, Tevatron and LHC to determine which 
decay channel has the highest statistical sensitivity. Secondly, for this particular channel, 
the theory prediction is compared to the observed experimental limits to conclude whether 
this sample is allowed or excluded at 95% CL. With the HiggsBounds, we find that most 
of samples (black bullets) with 180 < Ma < 330 GeV and 5 S tan < 10 have been 
excluded. Particularly, the latest CMS result of searching for H/A > 7+r~ has excluded 
most parameter space with a low M, and low to moderate values of tan 6, as shown in the 
upper left corner of Fig. [| We have checked that the current low energy constraints from 
B, —> utp” and B, + X,y are weaker than H/A —> rtr for our interested region (low to 
moderate values of tan 8). We note that the supersymmetric loop corrections generally lead 
to a contribution of the order of a few percent of the SM value. Hence, the suppressions in 
Higgs signal strengthes ju, and uyy» are mostly governed by the increase of the width of 
the lightest CP-even Higgs decay into bottom quarks and tau leptons at low values of m4. 

Fig. [I] suggests that all the additional non-SM-like Higgs bosons should be heavy, with 
masses larger than about 330 GeV. This is the commonly discussed decoupling limit of 
the MSSM. However, as discussed in , it is also possible to have the MSSM parameter 
conditions for alignment independent of decoupling, where the lightest CP-even Higgs boson 
has SM-like tree-level couplings to fermions and gauge bosons, independently of the non- 
standard Higgs boson masses. In the alignment region, sin(a — 3) ~ 1 and the bounds on 
the heavy Higgs bosons that arise from the measurements of h + VV may be relaxed. Such 
alignment conditions are associated with very SM-like htt coupling and tend to be restricted 
to values of tan @ of order 10 or larger within the MSSM Caled As will be shown below, 
for such a large value of tan 2, the supersymmetric contributions to the SM-like Higgs self- 
coupling are found to be small (at a percent level). Hence, even in the alignment condition, 
we cannot expect a large deviation of the Higgs self-coupling with the SM value. 

In Fig. 2] we project the samples surviving all the experimental constraints on the planes 
of ma and mz, versus tan 8. The ratio \3/°°"/A3 is always smaller than 1 because of 
the negative MSSM corrections to Higgs self-coupling loa, From the left panel it can be 
seen that in most part of the allowed parameter space (the blue triangles), the values of 


AMSSM [NIM (we use 3h to denote hhh) are larger than 0.99. When both tan and ma 
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FIG. 2: Scatter plots of the samples surviving all the experimental constraints, projected on the 


planes of tan 8 versus ma and mç. 


become small, \3/5°//)3™ gets smaller, which can minimally reach about 0.97 for our 


samples. The reasons for \}/5°™ being so close to A 


3 are the following: (1) The dominant 
MSSM contributions to Af/5S“/)\3™ are from the stop loops, while the Higgs coupling with 
the stops is proportional to 1/sin 8. So it leads to an overall enhancement factor 1/ sin? 8 
in the corrections when tan 8 is small. However, such a region is obviously not favored by 
the measured Higgs mass, which needs a large tan 3 to enhance the Higgs mass; (2) A light 
ma causes a large mixing between two CP-even Higgs bosons and can sizably change the 
Higgs couplings with the SM fermions. But as mentioned before, m, should be heavier than 
about 330 GeV to satisfy the experimental constraints in our scan. For the right panel it 
should be mentioned that since the small values of A37°°™/d3 occur in the small tan 8 
region, heavy stops are usually needed to enhance the Higgs mass through loop corrections. 

In Fig. B| we show the MSSM Higgs couplings in comparison with the SM predictions. 
The ILC (1 TeV, 1 ab~') sensitivities to the alteration of the couplings are also ploted, 
where the regions between the bars give too small alterations to be detectable at ILC. The 
HL-LHC (14 TeV, 3 ab~') sensitivities are much worse than ILC and are not shown here. 
From Fig. [B| we have the following observations: (1) In the MSSM, the Higgs gauge couplings 
and top-Higgs couplings are respectively changed by the factors sin(3 — a) and cosa/ sin 8, 
but the ratios CMS’ /ORM, and CMSSM /C5™ for our survived samples are very close to 


unity due to ma >> mz. So, even if these couplings can be measured at percent level at ILC, 
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FIG. 3: Same as Fig] but showing the Higgs couplings. The ILC (1 TeV, 1 ab~!) sensitivities 
to the alteration of the couplings are also ploted (the regions between the bars give too small 


alterations to be detectable at ILC). 


it can not constrain the MSSM parameter space with a small A}455M /A9M; (2) The ratio 


O / Cae is always smaller than one for our samples because a large mixing between the 
stops or heavy stops needed by the Higgs mass interferes destructively with the top loop and 


Ca is suppressed, while CMSSM /Ca™ 


Bay hy, can be greater than one due to the constructive 


contribution with the W loop. Although both couplings only slightly deviate from the SM 
predictions, the high precision measurements on C4,,,., at the ILC will be able to exclude 


some part of the parameter space with \}75°™ / ASM > 0.977. 


C. Results for the NMSSM (à < 0.7) 


In Fig. [4] we project the NMSSM (A < 0.7) samples allowed by the constraints (1)-(5) on 
the planes of A versus « and « versus the singlet component |Q2,| in the SM-like Higgs boson 
hə. Here we take 2mp, > mp, so that he will not decay into a pair of hı. We can see that 
when A and « approach to zero, tan 8 has to be large in order to enhance the Higgs mass. 
While in small tan 8 < 10 region, the values of À for most samples are larger (in magnitude) 
than «. Since the singlet component |Q2,| in the SM-like Higgs boson hs can potentially 
affect the Higgs self-coupling in Eq. (22), we also show the dependence of |Q2,|. Since the 
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FIG. 4: The NMSSM (à < 0.7) samples surviving all the experimental constraints, projected on 


the planes of \ versus « and the singlet component |Q2,| in the SM-like Higgs boson hə. 


singlet component |Q2,| in the SM-like Higgs boson hz can potentially affect the Higgs self- 
coupling in Eq. (22), we also show the dependence of |Q2,| on À and « for the NMSSM (with 
A < 0.7) samples which survive all the experimental constraints. It can be seen that |O2s5| 
deceases if both À and « go to zero. However, if A >> |k| ~ 0, which usually happens for 
small tan 8 < 10, |O2,| can be sizeable and lead to a large deviation from the SM prediction 
in triple Higgs boson coupling. For example, for A = 0.017 and k = —0.0038, Oo, ~ 0.215 
and Na SSM /\3M ~ 0.7. The reason is that in the singlet-doublet system, after decoupling 
h3 by requiring Mr, > 1 TeV, |O2s| is proportional to sin 0, where the mixing angle @ that 
determines the mixture of the singlet and the SM-like Higgs states can be approximately 
expressed as tan 20 ~ 2M33/(M5, — Ms) [35, 87]. Here, MZ are the CP-even Higgs mass 
matrix elements listed in Eqs. (1318). In case that A is not too small, a large 0 can happen 
when M3, — M33 ~ M3. This leads to a large singlet component in the SM-like Higgs state 
and hence a sizeable modification of the Higgs trilinear couplings. 

In Fig. [5| we display the dependence of Me ae versus À and tan £ for the NMSSM 
(A < 0.7). As mentioned above, due to the Higgs mass constraint, most of the allowed 
model samples tend to have large values of A. Furthermore, the ratio raw MSSM / ASM is not 
sensitive to the value of tan 8. We also show the dependence of A3 9SM /)\3 versus the 


singlet component |Q2,| in the SM-like Higgs boson hz and the lightest CP-even Higgs mass 
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FIG. 5: Same as Fig. [4] but showing the dependence of A SBM Fé A versus A and the singlet 
component |Q,| in hz, where mp, < 2mp,. The ILC(1 TeV, 1 ab~!) and HL-LHC(14 TeV, 3 ab~') 


sensitivities are also plotted (the region below each horizontal line is detectable). 


Mp, for the NMSSM (A < 0.7). Besides, we plot the expected ILC(1 TeV, 1 ab~') and HL- 
LHC(14 TeV, 3 ab~') sensitivities to Higgs self-coupling from the direct measurements 
We can see that A SSM /AŞM becomes small with the increase of the singlet component 
|O2;| and can minimally reach 0.1 in the allowed parameter space. Meanwhile, such a large 
mixing can make the mass of the lightest singlet-dominant CP-even Higgs boson h, close to 
the 125 GeV SM-like Higgs boson hg. 

In Table [I| we present the properties of two CP-even Higgs bosons hı and hg for such 
a benchmark point at 14 TeV LHC. We can see that the cross sections of the single hı 
production are close to those of SM-like hg because of the large doublet and singlet mixing 
components in both hı and hə. However, the branching ratio hı > VV(V = Z,Wy,g) 
is greatly suppressed by the increase of the partial width of hı —> bb. Thus, the observed 
production rate of gg > hı — VV is much smaller than that of SM-like hz. We also checked 
and found that although the cross section of gg > hı — 7*T~ can reach 2.06 pb, it is 
still smaller than the upper bound given by the current LHC searches for H/A > rt77 
. Besides, it should be mentioned that the sizable modification of the self-coupling of the 
SM-like Higgs boson hs always accompanies with the great changes in other Higgs couplings, 


which can be seen in Fig. [6] So, given the limited sensitivity of measuring the Higgs boson 


16 


TABLE I: A benchmark point for the NMSSM with \ < 0.7 (hg is the SM-like Higgs boson). The 


cross sections (pb) are calculated for LHC-14 TeV. 


a) [a Ga] PRET 
0.31 105.3 1461.9 -716.2 0.798 
a ee 


Brn, yy 
ma (GeV) 


self-couplings, we anticipate that the precision measurement of Higgs couplings to gauge 


bosons and fermions could test this scenario at the LHC and future colliders prior to the 


direct detection of triple or quartic Higgs couplings. 
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FIG. 6: Same as Fig. B] but for the NMSSM (A < 0.7). 


In Fig. [6] we show the Higgs couplings in the NMSSM (A < 0.7) surviving all the exper- 
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imental constraints. For comparison, we also show the discovery potential of the expected 
ILC(1 TeV, 1 ab™t) and HL-LHC(14 TeV, 3 ab~') 57. From this figure, we obtain the 
following observations: (1) Due to the singlet admixture in the SM-like Higgs boson, both 
Higgs gauge couplings and top-Higgs coupling can be maximally reduced by about 30% in 
the allowed region, which is much larger than in the MSSM. So, the expected measurements 
of the Higgs gauge couplings at the HL-LHC and ILC can exclude the parameter space 
with AGM SSM 13M < 0.82 and AJM SSM ASR < 0.93, respectively; (2) With the increase 
of the singlet component in the Higgs couplings, both Lo / Go and a (Gay 
are significantly reduced. On the other hand, due to the additional tree-level contribution 
(~ Avsin 28) and the positive mixing effect, we find that a stop with mass less than 200 GeV 
is still allowed by the SM-like Higgs mass constraint in the NMSSM (similar results have 
been obtained in previous NMSSM works lss). Consequently, the ratio Ge eer | Ga. be- 


comes larger than one, due to the constructive contribution from the light stop in loop. We 


approaches to AȘ}, CNMS5M / Gy can still be enhanced 


\NMSSM 
3h2 hoyy 


also note that even when 


by about 8%. 


D. Results for the NMSSM (à > 0.7) 


In Fig. [7| we display the dependence of ee /\3M versus tan and mp, for the 


NMSSM (A > 0.7). Similar to Fig. [4] the larger A is, the smaller tan 6 becomes to satisfy 
the requirement of the Higgs mass. The ratio AM SSM /)3™ can vary from -1.1 to 1.9 in our 
scan. For example, the ratio Ah SSM IASI is equal to 1.89 when A = 1.51 and «x = 0.67, 
and is —1.04 for \ = 1.57 and & = 1.16. Since we require mp, > 1 TeV, for our samples with 
Mhs > Mh» Aa SRM ASI is approximately proportional to à? and becomes large with the 
increase of A. In our scan ranges, such a feature could either enhance or suppress the Higgs 
self-coupling with respect to the SM prediction, and yield potentially large effects in Higgs 
pair production cross sections hd bd. However, in general, the two masses Mma, and mp, are 
virtually independent and the mixing patterns are complicated. It is worth mentioning that 
the value of Aj)/°°"/\3™ S 1 strongly relies on the mass of the next-to-lightest CP-even 
Higgs boson hg. To be specific, when «x becomes small (large), mp, is inclined to be light 


(heavy). If mp, is lighter (heavier) than about 400 GeV, AJ SSM ASP is smaller (larger) 


than unity for most samples. The properties of hə will be discussed in the following. We 
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FIG. 7: The NMSSM (A > 0.7) samples surviving all the experimental constraints, showing the 
dependence of Ag JARA versus tan 6 and Mph,. The ILC(1 TeV, 1 ab~!) and HL-LHC(14 


TeV, 3 ab™t}) sensitivities are also plotted. 


should mention that the large \ and « that produce the large deviation of AJM SSM /\3M 
jeopardize the perturbativity up to GUT scale. Thus, the new unknown strong dynamics 
will appear at some cut-off scale A. On the other hand, the constructions of high scale 
theory by adding vector-like matter can allow for a larger À value and relax the cutoff scale 
to high values im which is however beyond the scope of our study. 

In Fig. |8| we plot the couplings of hz with gauge bosons and 7 leptons, normalized to 
the SM values. We can see that the gauge coupling hV V is always suppressed due to 
the presence of singlet (s) and non-SM doublet (H) components in hg. If hg is singlet- 
like, the hyt+r~ coupling is suppressed as well, while if hy is non-SM doublet-like, hyt+7~ 
coupling can be enhanced by tan 8. The detailed mixing patterns of s and H in hg and 
its couplings have been thoroughly investigated in . We checked that the cross section 
gg — h2 + 7*77 for our samples is at least one order lower than the current direct search 
bound on the non-SM Higgs bosons bd 28). The main reason is that for a heavy Higgs 
boson A2, the new decay channels, such as h — hıhı l6ll, can be opened and the branching 
ratio of hə > 7*7* will be highly suppressed. 

In Table we present the properties of two CP-even Higgs bosons hı and hg for a 
benchmark point at 8 TeV LHC. We can see that all the SM branching ratios of ho are 
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FIG. 8: The NMSSM (A > 0.7) samples surviving all the experimental constraints, showing the 


dependence of the Higgs couplings versus Mps. 


reduced due to the opened decay mode hə — hıhı, which can reach 53.6% for mp, = 282.0 
GeV. Such a Higgs-to-Higgs decay will lead to a resonant SM-like di-Higgs bosons production 
pp — hy > hıhı at the LHC. A resonance feature in the h,h; invariant mass can be served 
as a smoking gun to search for the heavy Higgs boson hə. With one SM-like Higgs boson 
hı decaying to two photons and the other decaying to b-quarks, the resonant signal may be 
observable above the di-Higgs continuum background for mp, < 1 TeV at the HL-LHC lod 
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FIG. 9: Same as Fig. [B] but showing the couplings of hı. The HL-LHC(14 TeV, 3 ab~') and ILC(1 
TeV, 1 ab~*) sensitivities are also plotted. 
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TABLE II: A benchmark point for the NMSSM with A > 0.7 (hı is the SM-like Higgs boson). The 


cross sections (pb) are calculated for LHC-14 TeV. 


rev NC EMC 
1.54 0.75 3.06 474.00 1035.41 -644.30 -0.203 
mG 


Br hiogy 
ma (GeV) 
Pom | om | ow | om | om | oo 


Next, we present in Fig. [9]the couplings of the SM-like hı to weak gauge bosons and tau 
pair for the NMSSM (à > 0.7). To show their correlation with the Higgs self-coupling, we 
use the red color to highlight the points that satisfy eo SM AS — 1| > 0.3. It can be 
seen that the large Higgs self-couplings corrections correspond to the sizable shifts in Higgs 
couplings with gauge bosons and tau pair. Since the tree-level mass of the SM-like Higgs 
boson hı could easily exceed 125 GeV, hı is likely to have non-negligible singlet and/or 
non-SM doublet components, which makes its couplings deviate from the SM predictions. 
From this figure, it can be seen that Ch MSSM /O8M, is always less than unity since any 
singlet and/or non-SM doublet components in hı will make the Higgs couplings to weak 
gauge bosons smaller than the SM predictions. However, as mentioned above, if the next- 
to-dominant component in hı is the non-SM doublet, the Higgs coupling with the down- 
type fermions may be enhanced and larger than the SM predictions, such as the hyr*+7~ 
coupling. Therefore, the future measurements of C),,;+;- and C,,yy couplings can give 
strong constraints on the parameters space of the NMSSM with à > 0.7 and set limits on 
the Higgs self-coupling Ce : 

From the above discussions, we can see that the large deviation of the Higgs self-coupling 


Ash, for the NMSSM (A > 0.7) is always accompanied by other collider signatures, such 
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as a shift in the Higgs couplings and the production on resonance of the non-SM doublet 
hə. At the LHC, the resonant production of hz may be observed through the channels 
gg — ha > T+T, hyhy. However, the sensitivities of these channels strongly depend on the 
mass of hy and on its singlet and non-SM doublet components. If hə is dominantly singlet, 
both direct searches will not be powerful in probing our scenario at the LHC since all the 
hə couplings to SM particles are greatly reduced. Thus, the precision measurement of the 
observed 125 GeV Higgs boson couplings will play an unique role in probing this scenario 
at future colliders. If hg is dominantly doublet (non-SM), and if mp, < mp, < 2mp,, the 
process gg — hz — TtT” is still greatly suppressed due to the reduction of the coupling hott 
despite the fact that the coupling hgrt+t~ can be maximally enhanced by a factor of 2. On 
the other hand for 2mp, < mp,, the decay hy — hıhı is open and contributes to the cross 
section of pp — hıhı. Therefore, besides the Higgs coupling measurements, a resonance 
feature in the hıhı invariant mass or an excess in the inclusive h,h, production can be used 


to probe this model at the future LHC. 


IV. CONCLUSION 


We examined the currently allowed values of trilinear self-couplings of the SM-like 125 
GeV Higgs boson (h) in the MSSM and NMSSM after the LHC Run-1. Considering all 
the relevant experimental constraints, such as the Higgs data, the flavor constraints, the 
electroweak precision observables as well as the dark matter detections, we performed a 


scan over the parameter space of each model and obtained the following observations: 


e In the MSSM, the Higgs self-coupling is suppressed relative to the SM value. Such a 
suppression was found to be rather weak and the ratio AMSSM/)™ is above 0.97 due 


to the tightly constrained parameter space, cf. Figs. [I] and 2} 


e In the NMSSM with A < 0.7, we consider the case that the SM-like Higgs boson mass 
Mn, is less than twice of mp,, so that ha will not decay into a pair of hı. We found that 
the Higgs self-coupling was found to be likely suppressed and the ratio \NMSSM /)\SM° 
can be as low as 0.1 due to the large mixing between singlet and doublet Higgs bosons, 
cf. Fig. [5] In that case, the coupling of the SM-like Higgs boson (h2) to to W and 


Z bosons and top quark pairs are all suppressed as compared to the SM prediction. 
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On the other hand, its couplings to loop-induced processes, such as photon pairs or 
gluon pairs, can be enhanced, cf. Fig. [6] Given the limited sensitivity of measuring 
the Higgs boson self-couplings, we anticipate that the precision measurement of Higgs 
couplings to gauge bosons and fermions could test this scenario at the LHC and future 


colliders prior to the direct detection of triple or quartic Higgs couplings; 


e In the NMSSM with à > 0.7 (also called A-SUSY), the Higgs self-coupling can be 
greatly suppressed or enhanced relative to the SM value (the ratio ANMSSM/)3™ can 
vary from -1.1 to 1.9), cf. Fig. [7] when hı is taken as the SM-like Higgs boson. Its 
coupling to W and Z bosons are always suppressed as compared to the SM predictions. 
On the contrary, its couplings to tau pairs can be either enhanced or suppressed relative 
to the SM value, cf. Fig.{9] When Mma, > 2mzy,,, it is possible to observe a new resonance 
production in the di-Higgs boson channel at the LHC. While the couplings of hz to W 
and Z bosons are always suppressed, its coupling to tau pair can deviate largely from 


the SM value, depending on the mass of hg, cf. Fig. 


Since the NMSSM can give rather different values (compared with the SM) for the trilin- 
ear self-couplings of the SM-like Higgs boson, the future collider experiments like the high 
luminosity LHC or ILC can probe NMSSM through measuring the Higgs self-couplings. 
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